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I. INTRODUCTION 



A. STATEMENT OF THE PROBLEM 

In its role of Developing Agency for Naval aircraft weapons 
systems, including airframes, avionics equipment, the Naval Air 
Systems Command (NAVAIR) must maintain the capability to conduct 
test and evaluation (T&E) of the equipment it is developing. 

This has been accomplished through the establishment and develop- 
ment of a group of T&E field activities with differing responsi- 
bilities for the various areas of interest for which NAVAIR 
requires testing. The role of the T&E field activities is to 
provide the various groups at NAVAIR Headquarters with the 
testing services necessary to ensure that the various developmental 
programs are progressing satisfactorily and that they have a 
reasonable probability of success. Also, information must be 
provided to allow for correction of deficiencies as early as 
possible, particularly in advance of expensive production 
decisions. Testing after a production decision provides a form 
of quality assurance and helps to identify areas requiring 
further correction. 

Managing the operation of the field activities, internally 
and at the Headquarters level, in the current environment of 
weapon system acquisition is complex and difficult. Several 
issues have developed which were discussed in a meeting of the 
NAVAIR Civilian Management Board and the Middle Management 
Council at Charlottesville, Virginia, in November, 1977. Some 
of these issues are listed below. 
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1. There Is a lack of stated definitive "corporate" 
(NAVAIRSYSCOM) strategy related to roles of the field activities 
in the overall Naval Aviation Program. The absence of declared 
top level strategy makes it difficult to develop a unified 
corporate plan for field activity management and has resulted 

in a lack of clarity and definition in roles. 03 

2. There has been a succession of piecemeal efforts at 
dealing with many problems relating to field activity management 
and utilization (e.g., decentralization, distribution of civilian 
personnel, workloading, etc.) which result in operating in a 
reactive, non-integra ted, and non-systema tic mode. 03 

3. There are several inconsistent pressures effecting the 
effective and efficient management of field activities in the 
area of contracting for services (e.g., OME Circular A -76, 0MB 
Circular A-109* Congressional language cn FY-1978 contracting 
policies, MAT-08 policies on the in-house prosecution of R&D )• CP 

4. Issues relating to the modernity of the facilities and 
equipment of the field activities have not been adequately 
treated; maintenance and modernization of the physical plants 
and buildings are not treated in a pro-active mode. CP 

3. Uncertain institutional funding existed due to shortages, 
priority changes and unanticipated costs. 0-3 

6. Unproductive redundancy appeared to exist. Fostering 
this in part, was an inadequate communication process between 
the Naval Air Systems Command (NAVAIR) Headquarters and affected 
field activities. CO 

7. Different goals and values existed as a result of many 
varying perceptions and Influences as well as too many field 
activity bosses (structural, functional). 03 

8. Deviations from assigned functional roles occurred due 
in part to the ready availability of resources outside assigned 
mission/function areas. These resources and the related tasks 
were viewed as tending to change the basic reason for which the 
activity existed and for which it was created. co 

9. NAVAIR does not have adequate knowledge and control of 
workload (functions and tasks) performed by field activities. 

For example: 

a. Headquarters ha3 limited knowledge of the total work 
being performed by our field activities. 

b. The capabilities of our field activities are net 
adequately defined. 



9 



c. The Navy Lab Program causes problems of prioriti- 
zation and responsiveness due to the separation of administrative 
and technical management. 

d. Field activities have no clear and disciplined 
statement of roles, missions, responsibilities, and actual 
functions . 

e. It is difficult to tie responsibility /authority/ 
accountability to field activities for programs. 

f. Activities are performing functions for organizations 
other than NAVAIR, which interferes with our priorities. 

g. Classification of activities as currently designated 

is confusing, and in some cases inaccurate. Example: PMTC is 

not really just a T&E activity. 

10. Functions and facilities at some field activities 
(particularly RDT&E activities) duplicate and overlap each other. 
Because of duplication of capabilities, our field activities 
compete for the same kind of work. For example Tactical software 
systems are spread among several activities causing duplication 
and waste. To further compound the problem the Navy Lab Program 
tends to duplicate SYSCOM and other field efforts. £l] 

11. Management policies, systems and procedures are inadeauate. 

a. The separation of administrative and technical 
management in different groups within NAVAIR causes problems of 
prioritization and responsiveness. 

b. There is inadequate coordination within HO of field 
workload, funding and manpower resources and tasks. 

c. There is a critical imbalance between resources and 

tasks . 

d. We have no system to adjust ceilings at field 
activities or between field activities and Headquarters . 

e. We have no system to balance facility requirements 
and workload at field activities. 

f. NAVAIR has an inadequate workload control system 
and no means to discipline that system. 

g. NAVAIR field management policies are unclear. 

h. Headquarters provides field activities with inade- 
quate guidance. 

i. Our employees have set up Informal, inappropriate 
networks to allocate tasks to field activities. 03 
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Some of the Issues above are overlapping; however. It should 
be clear that a myriad of areas exist where change is reauired. 
Some of these are spoken of in the past tense and policies have 
changed since November 1977 .> which would affect the management 
of the field activities, but most of the issues still exist. 

In addition to the issues cited by the Civilian Management Board 
and the Middle Management Council there are concerns relative to 
the roles of the T&E field activities and their relationships 
with the Operational Test and Evaluation Force (OPTEVFOR), 
problems associated with the designation of most of the T&E 
field activities as components of the Major Range and Test 
Facilities Base, and problems in the application of the Uniform 
Funding Policy to T&E field activity operation. 

B. OBJECTIVES 

The objectives of this thesis are to identify areas in the 
management of the NAVAIR T&E field activities where improvements 
are needed and to recommend changes to improve the system. In 
the discussion that follows, these areas of concern will be 
treated separately followed by recommendations for improvement 
in each area. Recommendations are briefly restated in the 
Summary of Recommendations. For the reader familiar with the 
areas discussed, the recommendations are identified separately 
so that the detailed discussion may be avoided if desired. 

C . METHODS 

To accomplish the objectives stated above, it is first 
necessary to examine the environment in which NAVAIR and its 
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T&E field activities must operate. Policies and procedures 
that have been established for the management of the T&E field 
activities must be discussed. Finally, the devices which have 
been developed to aid in the management must be analyzed. 

To determine the environment in which NAVAIR operates, 
specifically with regard to T&E and field activity operations, 
a study of various publications was conducted. This was followed 
by an examination of the instructions and directives of NAVAIR, 
the Naval Material Command, and DoD which establish the policies 
and procedures for NAVAIR management of its T&E field activities. 
From this, a list of questions was generated to determine how 
closely actual operations adhered to the established policies 
and procedures. Responses to these questions were requested from 
personnel at NAVAIR Headquarters and the T&E field activities. 

It is important to note that this was not a survey because the 
questions did not lend themselves to simple multiple choice 
answers and only the few people closely related to the problem 
were asked to respond. Also, the questions were not uniformly 
applicable to all the respondents. The results of these efforts 
were analyzed and interpreted to allow the objectives of the 
thesis to be satisfied. 

D. BACKGROUND 

Although problems still exist, defense systems acquisition 
has improved greatly in the 1970' s. Prior to this time emphasis 
was on total package procurement; however, about 1970 it was 
realized, particularly by the Blue Ribbon Defense Panel, that 
this concept was not working. This was largely due to over 
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optimistic cost and performance estimates and a lack of compe- 
tition throughout the acquisition process. Competition existed 
only at the paper study stage but design and price competition 
were nearly non-existent. As a result of the recommendations of 
the Blue Ribbon Defense Panel and other groups, policies began 
to be developed which stressed demonstrated performance as a 
requirement for continued development. This was further 
emphasized in 1976 when Office of Management and Budget Circular 
A-109 stressed that "each agency acquiring major systems should 
provide strong checks and balances by ensuring adequate system 
test and evaluation." GO 

The policies of A-109 are implemented in the Department of 
Defense (DOD) by DOD Directive 5000.1 supported by 5000.2 and 
5000. 3> which is devoted to test and evaluation. DODD 5000.1 
states : 

Programs should be structured and resources allocated 
to ensure that the successful demonstration of. program 
objectives is the pacing activity. CO 

Since demonstration of program objectives will require testing 

it is obvious that test and evaluation in the allocation of 

defense resources has increased greatly in importance. Further 

evidence of this is the participation in all Defense Systems 

Acquisition Review Councils (DSARC) of the Deputy Undersecretary 

of Defense for Research and Engineering (Test and Evaluation) 

whose responsibility it is to report to the DSARC and to the 

Secretary of Defense on test planning and results. 

The increased importance of test and evaluation has not been 
without its problems. In its report the Blue Ribbon Defense 
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Panel recommended that each service establish an operational 
test and evaluation organization independent of both the system 
developers and the users. This has resulted in the establishment 
of the Operational Test and Evaluation Agency (OTEA) by the Army 
and of the Air Force Test and Evaluation Center (AFTEC) by the 
Air Force. The Navy already had its Operational Test and Evalua- 
tion Force (OPTEVFOR) . All report directly to the office of 
chief of staff of their services, assuring them of independence 
from the developers and the users. 

Over the past 10 years changes have been made in the acquisi- 
tion process which reduce concurrency and program risk. The 
price has been additional management reviews, an increase in 
required test and evaluation, more use of competitive prototyping 
all of which greatly increase acquisition cycle time and cost. 
Gansler has cited GJ the effect in that older systems such as 
the NIKE AJAX and the HAWK were fielded in 6 years and 5 years 
respectively; whereas, newer systems such as PATRIOT and AEGIS 
will take 19 and 18 years. This increase in acquisition cycle 
time, and the ensuing efforts to reduce it, will result in 
continual pressure to minimize the time spent on T&E and will 
directly impact the relationship between NAVAIR and its T&E 



field activities. 



II. THE ENVIRONMENT OF 



NAVAIR AND ITS T&E FIELD ACTIVITIES 

A. DEVELOPMENTAL VERSUS OPERATIONAL TEST AND EVALUATION 

With the formation of the operational test organiza tions , 
test and evaluation is now formally divided into two distinct 
areas. Developmental Test and Evaluation (DT&E) and Operational 
Test and Evaluation (OT&E). DT&E is that which is conducted by 
or for the Developing Agency. Its first purpose is as an 
integral part of the iterative development process. Testing is 
conducted to evaluate equipment design candidates and the results 
are used to modify the design. This is followed by further 
testing and modification. This process continues until the 
initial paper design has been converted to fully developed 
hardware. The second Important function of DT&E is to provide 
information on the progress of the system development. Performance 
is measured against a set of time-phased cost and performance goals 
and objectives to determine if the system costs and performance 
are at the levels they should be at a given point in the program. 
The results can be used as the basis for continuation or adjustment 
to the development program as necessary. A third function of 
DT&E is to help assure the project manager that he has a reasonable 
expectation of succeeding in the OT&E which follows. 

Early OT&E also aids in the design process. To the extent 
that it is conducted on developmental hardware it helps to 
identify operational deficiencies to allow for correction during 
the development process. It also allows the comparison of 
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system performance with goals and objectives to determine program 
status. The difference between OT&E and DT&E lies in the orien- 
tation towards operational as opposed to technical factors and 
the emphasis on predicting future performance rather than evalu- 
ating current performance. The primary purpose of T&E then, is 
to provide answers to the questions, "What is the program status?" 
and "What is the likely outcome of the program? " ca 

An important program consideration is the amount of scarce 
resources to be allocated to the T&E process. Since both time 
and money are consumed by the T&E process without visibly advancing 
the program, the temptation is to minimize T&E. The danger lies 
in fielding a system which is not technically or operationally 
suita ble . 

The problem is compounded by the fact that it is often 
difficult to categorize tests as operational or developmental. 
Policies have been established which discourage joint testing 
unless it is dictated overwhelmingly by the costs of conducting 
separate developmental and operational tests. The tendency, 
therefore, is towards duplication of testing. Since the opera- 
tional test agencies have no obligation to minimize overall 
program costs or maintain the program schedule, there might be 
a tendency to over test and to be over critical in their evalu- 
ation of the results to aggrandize their own position in the 
acquisition process. In the case of developmental testing the 
situation is similar but slightly better. The developmental 
testers work for the program manager and are dependent on him 
for their funding; therefore, he has discretion in the amount of 
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funds and time allocated to developmental testing. The dilemma 
of the program manager is to decide how much time and money to 
spend on developmental testing. The temptation is to over- 
emphasize operational testing since he has little discretion in 
this area, thereby minimizing his testing costs. The danger in 
that approach is that a system that has not had the benefit of 
sufficient test and modification of the iterative developmental 
testing process may be killed by the sometimes overzealous 
operational testers. 

The culmination of OT&E is the OPEVAL which must be completed 
prior to approval for full scale production. This is preceded 
by a Certification of Readiness for OPEVAL which must be provided 
to ensure that the system to be tested is sufficiently represen- 
tative of the system that will ultimately be deployed. Procedures 
for the Certification are provided by NAVAIRINST 39^0. 2A and 
include a number of actions required by the various NAVAIR com- 
ponents under the overall coordination of AIR-05. In theory the 
process is relatively straightforward. In practice it has been 
a significant problem area. Problems have arisen in determining 
that the systems proposed for OPEVAL are representative of 
production systems. Because the DT&E inevitably points up 
deficiencies which must be corrected in production, major 
differences exist between the full scale development models and 
the production items. Correction of these deficiencies and 
configuration differences prior to OPEVAL is often time consuming 
and costly. A further requirement for OPEVAL is that logistic 
support, manning, and training for the operation of the system 
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during OPEVAL be representative of fleet deployment. This is 
often not possible without considerable delay because development 
of support equipment, training of personnel, and availability of 
spares in the supply system lag significantly behind system 
development. Readiness for OPEVAL, therefore, becomes a process 
of negotiation rather than a check-off of tne elements as intended. 
This is a delicate process for the program manager, since he must 
balance program schedule and cost constraints against the risks 
of failing to satisfy OPEVAL requirements for lack of some of 
the items mentioned above. 

One of the primary objectives of OT&E, and particularly the 
OPEVAL, is that it be conducted under conditions representative 
of those to be expected in field operations. The reasons for 
this are obvious; however, in practice it is often difficult to 
achieve for the following reasons: 

1. Inadequate threat environment. 

The operating conditions include the enemy weapons 
systems and their operational doctrine against which the system 
under test must operate. Simulations of these systems do exist, 
adequate for developmental testing; however, large scale networks 
adequate to simulate the complete operational environment are 
extremely expensive and do not exist. 

2. Insufficient numbers of test systems. 

For the same reasons it is desirable to operate against 
a large scale threat environment, it is also desirable to operate 
the system under test in conjunction with all the other systems 
it is expected to live with and in the manner it is expected to 
be operated. This is also extremely expensive and not generally 
done except for individual compa tability tests. 

3. Inadequate training of personnel. 

The objective is to conduct the test utilising personnel 
that will operate the system in the field. Since schools have 
usually not been established prior to a production decision, 
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adequately trained personnel do not exist. The systems must be 
maintained by persons of higher skill level than the operators, 
usually contractor personnel. 

4. System immaturity. 

In today's environment emphasis is placed on reliability. 
The systems tested; however, have been built by different tech- 
niques than will be used in full scale production. They will 
suffer from "infant mortality" problems which the mature system 
will not have. Also, with the limited number of systems and 
operating hours available, a statistically valid estimate of 
reliability usually cannot be determined. 

5. Lack of production configured support equipment and 
publications . 

This is related to the maintenance personnel problem. 

The support equipment designs and the publications cannot be 
determined until the production configuration of the prime 
equipment is fairly well defined. Therefore, they are usually 
not ready for testing in time and modified general purpose test 
equipment must be used. 

For obvious reasons it is desirable that objectivity and 
Independence be maximized in the test program. This is particu- 
larly true of the OT&E. Since the DT&E is conducted under the 
direction of the Developing Agency, biases are almost certain to 
be present. Evaluation of the results is often difficult under 
these conditions. In the OT&E the danger exists that "nice to 
have" features will be interpreted as necessary since OT&E is 
generally conducted without regard to system cost. Another less 
likely possibility is to minimize deficiencies in seeking early 
deployment. Factors which alleviate this problem are the checks 
and balances between the DT&E and OT&E. 

Previously, emphasis on OT&E was at program stages just 
prior to the production decision because of the large funding 
commitment at that point. Lately it has been realized that 
emphasis on earlier OT&E is needed for four reasons. 
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1. In the early stages of system concept formulation it is 
necessary to know that the concepts are tactically sound. 

2. To have any impact on system development OT&E must be 
conducted earlier than just prior to the production decision 
since by then most of the research and development resources 
have been expended. 

3. Cost and schedule impact can be minimized by early 
detection and correction of operational deficiencies. 

4. Commitment of funds for limited production prior to the 
major production decision requires earlier availability of test 
results . CO 

The role of OPTEVFOR is defined in Reference 6. It directs 
OPTEVFOR involvement in all phases of the acquisition cycle and 
provides for close coordination between OPTEVFOR and the developing 
agency. Specifica lly, during the program validation phase OPTEVFOR 
will : 



Establish liaison with the developing agency for 
providing operational planning assistance in determining 
the testing required for the current project, anticipated 
future testing requirements, and defining critical opera- 
tional issues. DU 

Participate in test planning and independently observe 
selected tests and demonstrations conducted by the develop- 
ing agency, and examine test results. submit an assess- 

ment of this initial test and evaluation to the Chief of 
Naval Operations, and the developing agency, including 
comments and recommendations concerning future operational 
suitability of the system, progress to date, and operational 
issues requiring further examination. DU 

During the full scale engineering development phase OPTEVFOR 
will : 



Independently observe selected developmental tests 
and view the data from an operational viewpoint, to 
verify readiness for operational testing. X?1 

From this it can be seen that much of the work of the T&E 

field activities will be evaluated by OPTEVFOR; therefore, it is 

necessary to be continually aware of their opinions and priorities. 
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Test programs may have to be adjusted to help satisfy OPTEVFOR 
requirements. Since ultimately responsibility for responding to 
criticisms of the test programs will fall to the program manager 
and his team, they should closely monitor the planning, conduct, 
and results of the tests conducted by the field activities. This 
will be discussed further in a later section. 

B. THE MAJOR RANGE AND TEST FACILITY BASE 

In 1970 the Blue Ribbon Defense Panel recognized the need 
for better test and evaluation management and recommended a major 
DOD-wide study of test and evaluation facilities. The study was 
completed in August 1971 and resulted in the formation of the 
Major Range and Test Facility Base (MRTFB), which consisted of 
26 T&E activities of the Army, Navy, and Air Force with a combined 
annual budget in excess of $1 billion. £7] The activities 
currently comprising the MRTFB are listed in Appendix A. 

Policies for the use, management and operation of MRTFB 
activities are established in Reference 8. The mission of the 
MRTFB is to: 

Provide a broad range and test support base to all DOD 
Components responsible for development, test, evaluation 
and operation, as applicable, of material and weapon systems, 
and to other Federal Government Agencies having need for 

that support. E7 

Overall responsibility within DOD is assigned to the Deputy 
Undersecretary of Defense for Research and Engineering, Test 
and Evaluation (DD(T&E)). Authorities and responsibilities for 
DD ( T&E ) , Secretaries of the military departments. Facility 
commanders, and Facility users are established. Reference 9 
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assigns responsibility for management of Navy portions of the 
MRTFB to the NAVAIR Assistant Commander for Test and Evaluation 
(AIR-06) . 

C. THE UNIFORM FUNDING POLICY 

Prior to fiscal year 1975 no single policy existed for 
assessing the costs of operation of test facilities. Some 
activities were funded institutionally. That is, money for 
their operations was provided from a separate appropriation 
independent of any program utilizing the test facilities. A 
second method, industrial funding, required that facility users 
pay all the costs, including overhead, of operating a facility. 

A third policy required that users pay only the direct costs 
associated with their tests, with the remainder of costs to be 
paid from institutional funds. There were also combinations of 
these methods which sometimes varied depending on who was the 
user . 

The study completed in August 1971> noted that because of 
the differences in funding policy, test site selection was often 
based on cost to the user instead of test site capabilities. As 
a result, in January 1973> the Deputy Secretary of Defense 
directed the establishment of a Uniform Funding Policy at 18 of 
the 2 6 MRTFB activities, with users paying direct costs and 
indirect costs being paid from institutional funds. The policy 
was incorporated into DODD 3200.11 and went into effect in 
fiscal year 1975. The Navy activities to which the Uniform 
Funding Policy applies are the Naval Air Propulsion Center, 
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Naval Air Test Center, Naval Weapons Center, Pacific Missile 
Test Center, and the Atlantic Undersea Test and Evaluation 
Center . 

DODD 3200.11 states that the intent of the Uniform Funding 
Policy as follows: 

1. Provide for interservice compa tability , efficiency and 
equity, without influencing technical decisions on testing, nor 
inhibiting legitimate and valid testing. 

2. Reimbursement for use of the ranges and test facilities 
will be required. 

3. All the costs of the range or test facilities which are 
not reimbursed by users will be funded by the management agency 
of the particular range or facility. 

4. All DOD Components and other Federal Government agencies 

will reimburse the ranges and test facilities for direct 

costs, excluding military labor. 

5. Direct Costs. Those expenses which can be immediately 
and directly identified with a specific User program (usually 
documented by a job order). These costs Include direct labor, 
direct material, minor construction, special purpose equipment 
and other like costs. They include all such expenses that can 
with reasonable effort be identified consistently and uniformly 
to specific User programs. The direct cost for labor Includes 

a load factor covering items such as leave and contributions to 
the retirement program. £8J 

In concept the Uniform Funding Policy is relatively simple. 
The implementation, however, has led to problems. Historically 
it has been difficult to predict the workload of test activities 
with any accuracy, even for the following fiscal year. Without 
an accurate estimate of direct funding it is impossible to 
accurately budget the institutional funds required. Expenditures 
for civilian personnel are relatively fixed and priority for use 
of institutional funds must be in this area. The impact is on 
capital expenditures for new test capabilities. An orderly 
program Is difficult to establish because funds must be held in 
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escrow against the possible shortfall of direct funding. This 
will be discussed in a later section. 

Problems also result for the users of the test facilities 
because of program delays. If the user plans a test for a given 
fiscal year, allocates the money to the test activity, and because 
of system problems the test is delayed beyond the end of the 
fiscal year, the program manager finds himself at the end of 
the fiscal year with the allocated test funds he can no longer 
use and he still must provide test funding out of the following 
year's budget. 

Increases in administrative workload as a result of the 
implementation of the Uniform Funding Policy have been docu- 
mented . CO These increases were due largely to the increased 
budgeting requirements and the need for more detailed estimates 
and reports. The increased workload has largely been absorbed 
with existing, already limited, personnel resources. The increased 
workload was not limited to comptroller personnel, but also 
affected technical personnel and detracted from their attention 
to their technical duties. 

D. THE BUDGET PROCESS 

1 . Discussion 

The process by which NAVAIR must prepare budgets for 
T&E field activity operations is complicated by the fact that 
they are funded from multiple appropriations. Institutional 
funding is provided from the Category V, Management and Support, 
funds. User funding may come from Research, Development, Test 
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and Evaluation, Navy (KDT&EN) funds or from one of the procurement 
accounts, depending on the actual project being prosecuted. The 
problem, then, becomes one of balancing the amounts budgeted from 
each appropriation to avoid violation of Section 3678 of the U.S. 
Code which states that funds must only be spent for the purpose 
for which they are appropriated. Reference 10 states: 

Apportionment of funds includes an amount for unfunded 
reimbursements expected to be earned during the fiscal 
year. The allocation of these anticipated reimbursements 
must not exceed the expected year-end gross unfunded 

accounts receivable. Work supported by reimbursable 

orders is to be financed within the subhead under which 
the order is accepted. 

Budget estimates for Navy procurement appropriations are 
submitted by NAVAIR to NAVMAT where they are consolidated and 
submitted to the Office of Budget and Reports/Fiscal Management 
Division which is responsible for formulation of the total Navy 
budget. Budget estimates for the RDT&EN appropriation are 
submitted by NAVAIR, usually by a different office than for the 
procurement appropriations, to the Deputy Chief of Naval Material 
for Development where they are coordinated and submitted to the 
Office of Naval Research for consolidation and ultimate submission 
to the Office of Budget and Reports/Fiscal Management Division. 

The separation of budget responsibilities makes consolidation 
of the various appropriations budgets into a single budget 
estimate for T&E field activity operations difficult, if not 
impossible. Paragraph 7 of Reference 11 requires that NAVAIR 
users of the T&E field activities submit written fiscal planning 
information to AIR-6103, the T&E Field Activity Support Branch 
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of AIR-06. This would help alleviate the problem; however, 
responses to questions do not indicate that the policy is being 
followed . 

Another problem in the budgeting process is the time limita 
tion on the availability of funds. Congress enacts FLDT&EN 
appropriations for two years but Navy policy is "to program 
RDT&E effort on an annual incremental funding basis as opposed 
to the fully funded basis on which procurement programs are 
developed . " DO The Navy Comptroller Manual states: 

Although the RDT&EN appropriation is legally available 
for obligation for 2 fiscal years, the objective is to 
use these funds during the initial year of availability. 

The budget should be formulated to reflect this funding 

objective. However, if the award of a planned new 

contract, or the issuance of a reimbursable order, is 

not possible until the second year of availability, 

the preferred action is to restructure the funding plan 
to utilize the original funding in the first year of 
availability for urgent deficiencies elsewnere in the 
program and to use the following year's appropriation 
to fund new contracts or orders. DO 

This policy exacerbates the problem mentioned earlier in 
reference to the Uniform Funding Policy. Program managers need 
not lose their funds due to program slippages if the money is 
legally available for another year. 

2. Recommendations 



The policy requiring users of the T&E field activities 
to submit written fiscal planning information to AIR-6103, if 
followed, would improve the ability to prepare accurate consoli 
dated budget estimates for TScE field activity operations. 
Adherence to this policy should be further emphasized as it 
would help give NAVAIR managers a better grasp of the overall 
costs of T&E in the acquisition process. 
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The policy which requires obligation of all of the 
RDT&EN funds of a fiscal year appropriation within tha first 
year of availability should be reexamined. There may be more 
advantages than disadvantages in allowing the money to be 
obligated throughout the full two years it is available. 

E. MAJOR AND LES3-THAN-MAJ0R SYSTEM ACQUISITIONS 

Policies and procedures for major systems acquisitions have 
been well established and documented at length. This area has 
received considerable attention because of the large sums of 
money involved. For less-than-ma jor systems the sum involved 
in a single acquisition is smaller and, therefore, of less 
importance. Nevertheless, when all of the less-than-ma jor 
systems acquisitions are considered in aggregate the amount 
involved is not trivial. Unfortunately, for these systems she 
procedures have not been as well established, particularly with 
regard to establishing T&E requirements and planning to satisfy 
those requirements. 

One reason for the problem is the volatility of the acquisition 
process. DOD Directive 5000.1, the document that prescribes 
policies and procedures for major system acquisition, has teen 
rewritten twice within the last five years and is currently 
being rewritten again. Below the level of DODD 5000.1 there are 
numerous instructions and directives which implement the broad 
policy guidelines. When the major policy document changes as 
rapidly as it has, these subordinate documents almost never 
catch up. This results in directives which are often in conflict 
with each other and It is often impossible to be in compliance 
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with all the current Instructions. Also, programs are generally 
of such duration that they outlast changes in policy. This is 
particularly true when it is realized that many programs consist 
of derivatives of existing programs and, therefore, do not fit 
into the current phases of the acquisition process. 

Policies and procedures for test and evaluation established 
by higher authority are implemented within NAVAIR by NAVAIRINST 
3960. 2A which states: 

The provisions of this instruction apply to the following: 

All acquisition projects including those classified as 
Acquisition Categories (ACAT) I, II, III, and IV projects. 

Projects involving significant modifications to existing 
systems and equipment whose Research, Development, Test and 
Evaluation (RDT&E) and Procurement Appropriation Funding 
Requirements and other criteria would qualify them as ACAT 
I, II, III, or IV projects. 

Individual subsystems, equipment or components developed 
as a part of a major system such as a radar equipment, a computer, 
acoustic processor, etc. which are susceptible to separate T&E 
and which are required by competent authority to be the subject 
of a separate Test and Evaluation Master Plan (TEMP) or Test and 
Evaluation Plan (TEP). 

"Off-the-shelf" commercial systems and equipments or signi- 
ficant modifications thereof when the program decision authority 
approves this alternative in lieu of new development to meet an 
established mission need. 

Individual subsystems, equipment or components which are not 
developed as a part of a major weapon system and which are 
susceptible to separate T&E and are required by competent 
authority to be the subject of a separate TEMP to TEP. 0L2J 

Definitions of the Acquisition Categories are given in 
Appendix B. The TEMP and TEP will be discussed in a later 
section. From the statement above and the definition of the 
Acquisition Categories it can be seen chat it is unlikely that 
the provisions of a formal T&E program would not apply to most 
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acquisitions. NAVAIRINST 3$60.2A is relatively new and it will 
take some time to determine if the policies are followed for all 
acquisition projects; however, if it is applied, it will go a 
long way towards improving the budgeting of T&E field activity 
operations. This is especially important in today's environment 
of conversions in lieu of procurement and service life extension 
programs . 

F. THE TEST AND EVALUATION MASTER PLAN 

To help make the T&E of major systems an orderly process, 
the program manager is required by OPNAVINST 3960.10 to submit 
a Test and Evaluation Master Plan (TEMP). The TEMP is the 
controlling T&E planning document prepared by NAVAIR in collabor- 
ation with NAVMAT, OPTEVFOR, and the Board of Inspection and 
Survey, when applicable, and is approved by OPNAV. 12 It 
serves as a contract between the Developing Agency and the user 
for the T&E to be conducted throughout the program. Included 
are information on program goals and milestones, system parameters, 
performance and cost thresholds, and test resource requirements. 

The test schedule and the level of testing to be conducted are 
keyed to the major project milestones. Separate sections are 
provided for DT&E and OT&E. The DT&E section is prepared by 
NAVAIR and the OT&E section is prepared by OPTEVFOR. Emphasis 
is placed on resolving issues prior to submittal to OPNAV; 
however, sometimes this is not possible and the TEMF must be 
submitted with open issues for OPNAV resolution. 
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The TEP is the controlling T&E planning document for ACAT 
IV projects and is approved by the Deputy Commander, NAVAIR. 

It is prepared within NAVAIR and contains the same type of 
planning information contained in the TEMP for larger programs. 

Prior to seeking a Milestone I decision the program manager 
is required to have the T&E plan prepared. For major programs 
it is required as an annex to the Decision Coordinating Paper, 
the document used as the basis for the Defense Systems Acquisition 
Review Council recommendation and the Secretary of Defense decision. 
The initial version is to avoid specifics of the test program 
and concentrate on the broad test plan and the critical tests 
required for satisfying major project objectives. Also, it is 
to identify the total resource needs for the test program in 
terms of manpower, material, facilities and funding. The plan 
is to be revised during the life cycle to add details as they 
become known. The revision prior to the Milestone II decision 
should contain sufficient test program specifics to ensure that 
the planned testing is sufficient to demonstrate program 
requirements . 

Within NAVAIR Headquarters AIR-05, the Systems and Engineering 
Group, is responsible for the technical content of the T&E plan 
and technical direction of the T&E program. AIR-06 is responsible 
for integrating the various elements, coordinating the staff 
review of documents, coordinating with the T&E field activities 
and arranging for the required AIR-06 managed resources. 

The use of T&E planning documents has the potential to greatly 
improve the forecasting of user requirements and funding for the 
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T&E field activities and to aid in overall forecasting of field 
activity workload and requirement for institutional funding. 

The effectiveness and efficiency of the TEMP process has not 
been optimized, however, because it has not been sufficiently 
integrated with the other information systems available to 
provide a composite picture for all projects and all field 
activities . 
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III. THE RELATIONSHIP OF THE ELEMENTS 
OF T&E FIELD ACTIVITY CONTROL 

A. APPLICABLE DIRECTIVES AND INSTRUCTIONS 

A list of most of the major directives and instructions 
applicable to the structure and management of NAVAIR T&E programs 
and the operation of the T&E field activities is included in 
Appendix C. 



B. DIVISIONS OF RESPONSIBILITY 
1. Discussion 



Policies and procedures for planning and conducting 
NAVAIR T&E programs are contained in NAVAIR Instruction 3960. 2A. 
Overall project responsibility is delegated to the program 
managers (PMA) for the individual programs; however, the primary 
functional responsibilities for T&E are delegated to AIR-06 and 
AIR-05, the Systems and Engineering Group. Figures 1 and 2 show 
the relationship of NAVAIR with external organizations and the 
relationship of the various elements within NAVAIR. Guctir.g 
from 3960. 2A: 

The MATACQ Group (AIR-05) is line responsible for the 
performance of the full set of acquisition functions 
within the matrix project management concept except for 
those specific program areas assigned to other Headquarters 
Groups (e.g., Target Systems). Accordingly, Air-05 is 
responsible for coordination of the implementation of the 
comprehensive T&E programs and the planning, execution, 
and technical evaluation of the DT&E and PAT&E (except for 
BIS Acceptance Trials) elements thereof. The T&E Group 
(AIR-06) is the designated single point of contact for 
NAVAIR T&E program matters and as such is line responsible 
for the interpretation and implementation of all T&E 
policies and procedures. AIR-Ob functions to develop 
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Figure 1 

External Organizational Relationships 
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Internal Organizational Relationships 




and promulgate appropriate NAVAIR policies and procedures 
to provide T&E guidance and assista r.ce to all organizational 
units within NAVAIR. AIR-06 also provides NAVAIR and Navy- 
wide management of specifically assigned NAVAIR T&E field 
activities and Navy elements of the DOD Major Range and 
Test Facility Base (MRTFB). 

(The title of AIR-05 was changed from Material Acquisition Group 
to Systems and Engineering Group on 2 April 1979.) 

Significant responsibilities of AIR-05 and AIR -06 in 
the control of field activity operations, as specified in NAVAIR 
Instruction 3960.2A are listed below. 

For AIR-05* 

In collaboration with AIR -06 participate in the surveys 

of candidate T&E field activities to assess their individual 

availability, capability, and applicability to accomplish the 
planned T&E programs . 

Ensure coordination with AIR -06 on matters of T&E 

policy, T&E resource requirements, selection of supporting T&E 
field activities and contractors and funding estimates for 
support of the DT&E and PAT&E programs. 

Prepare and coordinate with AIR-06, the AIRTASK and 
contract work assignments for NAVAIR T&E field activities and 
contractors . 

For AIR-06: 

Provide executive management for all of the NAVAIR T&E 
field activities. 

Maintain working interfaces with all available T&E 
activities including Navy, Army, and Air Force elements of the 
DOD MRTFB, as well as the Navy laboratories and appropriate 
industrial concerns, in order to establish an awareness and 
understanding of the T&E resources available for selection to 
support NAVAIR T&E programs. 

Conduct surveys for the selection of field activity 
sites for the conduct of specific acquisition project T&E 
programs . 



Approve test site selections . 

Render T&E program support and assistance to PMA's in 

the following: 
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(d) Expediting T&E reports from the T&E field 
activities . 

Provide T&E representation to the NAVAIR Change Control 

Boards to enable planning for T&E resource support for the 

T&E of the modification programs at the AIR-06 managed field 
activities . 

Responsibilities of the T&E field activities are as follows: 

Provide assistance to AIR-05 and to OPTEVFOR and BIS 
when appropriate in structuring the T&E programs for acquisition 
projects and in planning the elements thereof. 

Provide budgetary estimates for the accomplishment of 
T&E workload scheduled for their respective activities. 

Identify T&E resources required to support approved 
T&E plans (TEMP's and TEP 1 s ) . 

Provide representatives for site survey teams when 
requested by NAVAIRHQ (AIR-06). 

Perform T&E directed by NAVAIR AIRTASK/Vork Unit 
Assignments . 

Provide test reports on a timely basis to support the 
project decision making process at milestone points identified 
in the approved test plan. 

Advise NAVAIR (AIR-05/06) promptly of actual or 
threatened slippages in the completion of work assignments 
which may impact the scheduled key project milestones shown 
in the approved test plan and provide recommendations for 
dealing with the causes thereof. 

Advise NAVAIR (AIR-05/06) promptly when test results 
indicate major deficiencies which will require system or 
equipment redesign to enable meeting the T&E program objectives 
stated in the applicable TEMP or TEP or which are likely to 
result in the breaching of performance, schedule and funding 
thresholds stated in the applicable DCP/NDCP. 

Responsibility for introducing the controlling workload 
at Navy MRTFB activities (the T&E field activities plus portions 
of other Naval Material Command activities) is assigned to AIR-06 
by NAVMAT Instruction 3960.10. As such, they are required to 
perform workload management and be responsive to the technical 
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guidance of workload sponsors (primarily AIR-05). V.'orkload 
sponsors are required to provide timely notification of workload 
assignments to activity commanders to ensure proper support. 

The responsibilities of various organizations and the flow of 
documentation required are shown in Figures 3 and 4. M 
Figure 3 applies to workload sponsored by NAVAIR; whereas. 

Figure 4 applies to workload sponsored by organizations external 
to NAVAIR. It should be noted that the project estimate is 
broken into direct and institutional funds. To provide for 
advanced planning the instruction requires that fiscal planning 
information for the next five fiscal years be provided to AIR-06 
by 1 May of each year. The MRTFB activities are to submit 
workload plans for all work assigned. As was mentioned in the 
section on the Uniform Funding Policy, it is difficult to 
forecast the work of a T&E activity even one year in advance. 
Forecasting for five years is extremely imprecise. This is 
particularly true because the forecasting is being done oy the 
T&E activity itself rather than the users. Even knowing what 
programs are scheduled for the next five years, it is often net 
possible to predict requirements the users will generate for 
T&E support. 

In Figures 3 and 4, the portion of the project estimates 
submitted by the field activities for institutional funds are 
used primarily for improving facilities and equipment. Unfortun- 
ately, no formalized provisions exist for tying these elements 
together into a coordinated plan for updating the facilities of 
the field activities separately or collectively . An Improvement 
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Workload Flow Chart 




and Modernization (I&M) program has been established by AIR-06 
in an attempt to improve this situation. Various efforts such 
as Project 20 have required the field activities to project their 
facilities and equipment improvement needs so that planning for 
these requirements can be done in a centralized, logical manner. 
Provisions for incorporating into these projections requirements 
for general facilities improvements that are not directly tied 
to projects have been inadequate. Funding in this area has been 
very uncertain, probably due to the fact that insufficient 
justifying information is available to secure the funds. Institu- 
tional funds for this purpose are usually what is left after all 
other needs have been satisfied and are inevitably insufficient. 

2 . Recommends tions 

To improve the ability to plan and budget the workload 
of the field activities, closer dialogue between the activities 
and the users is recommended. The T&E field activities should 
be considered to be and treated as a part of the NAVAIR team, 
not subject to the arm's-length relationship required with 
contractors . 

To provide for required updating of facilities at the 
field activities, efforts should continue for a strong I&M 
program. This should include both direct, project related, and 
general requirements. 

C. THE FIELD ACTIVITIES 

Management guidance at the field activities is primarily 
concerned with the preparation of aggregate information on 
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operating budgets and workload. Another area of concern is 
assignment of personnel ceilings. Project work is nominally 
assigned to a single point of contact at the field activity. 

In actuality, however, the primary project Interface is between 
the operating personnel at the field activity and the "users" 
at Headquarters. Because of this, the operating people are 
serving two different, and often conflicting, masters. The 
degree to which this is a problem probably varies between field 
activities for a variety of reasons, but particularly the size 
of the organization and the degree to which management and 
operating people are separated. A result of the disconnect 
between the management and operating personnel is that while 
management is seeking additional resources to perform existing 
work, operating personnel are often seeking additional work for 
existing resources. 

D. PRIORITY SYSTEMS 
1. Discussion 

Currently, there is no overall priority system for 
NAVAIR. Field activities establish their own priorities based 
on informal guidance from various sources and their own percep- 
tions of what is most important. This has the advantage of 
allowing them the flexibility to respond to situations based on 
first-hand knowledge of the situation. Efficient scheduling is 
allowed without micromanagement from Headauarters . On the other 
hand, under this system, priorities tend to be more dependent 
on the dollar value of the test program than on its overall 
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urgency and value to fleet operations and National defense. 

> 

Also, local establishment of priorities may tend to ignore Life 
Cycle considerations and the requirement to complete certain 
testing prior to a scheduled Milestone Decision. 

2. Recommendation 

To date, the absence of an overall priority system has 
not resulted in any major problems. This is primarily because 
no major program has been delayed significantly due to lack of 
T&E resources. As the requirements for T&E programs become more 
formalized and the length and complexity increase this situation 
can be expected to change. Reductions in resources will exacerbate 
the situation and delays will be incurred. When this happens it 
will be imperative that a single priority system is able to 
establish the order of precedence of the various programs. The 
problem has been recognized and AIR-610 has been working with 
AIR -620 and AIR-59 to develop a priority management process. 

These efforts should continue until a workable system has been 
devised and implemented. 

E. PERSONNEL REDUCTIONS 

Because of policy changes in recent years a continuing 
pressure for reduced personnel at Headauarters and at the field 
activities has developed. The Office of Management and Budget 
in Circular A -76 stressed the contracting out of work to the 
private sector when practical. Congressional pressures to reduce 
the average grade level of civilian employees has been increasing 
in recent years. In response to these pressures the level of 
review for filling of jobs, particularly at the middle and upper 
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management levels, has been raised. Quotas on the filling of 
vacancies have been established and several positions have been 
downgraded. The result has been that the number of positions 
occupied has decreased much more rapidly than celling point 
reductions would indicate. Furthermore, all of this is occurring 
in the face of Increasing complexity of systems being acquired 
and T&E requirements for these systems. Since little relief of 
the situation can be expected in the near future, the only hope 
for solution is to use the remaining resources more efficiently 
through even greater centralization of controls to Headquarters. 
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IV. DOCUMENTATION AND INFORMATION SYSTEMS 



A. AIRTASKS AND WORK UNIT ASSIGNMENTS 
1 . Discussion 

Project work is currently assigned to the field activity 
by Airtask. The policy and procedures for AIRTASKS are established 
by NAVAIRINST 3900.8 a. £ 13 ] The AIRTASK, generally no more than 
two pages in length, represents an outline of the work to be 
performed. It includes background information, detailed require- 
ments, source and disposition of project equipment and aircraft, 
reports required, and a funding estimate. The background 
information discusses events leading to the project and estab 
the need. The detailed requirements provide a list of the 
specific functions to be performed. These are usually very broad 
and provide maximum flexibility for interpretation as the project 
progresses and the perception of the details of the requirements 
change. The source and disposition of project equipment and 
aircraft usually only states that existing assets should be used, 
that they will be provided, or that requirements should be 
coordinated with the NAVAIRSYSCOM Aircraft Custodian. The reports 
required may vary greatly but usually include, as a minimum, a 
final report and recently, a detailed project plan to be submitred 
early in the prosecution of the project. The funding estimate 
states the amount of money to be provided for the project by 
separate correspondence, usually a work request, from Comptroller 
to Comptroller. 
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Work Unit Assignments are subordinate elements of an 
AIRTASK for the performance of a specific task within the scope 
of the previously assigned AIRTASK. The same form (NAVAIR 3930/1) 
is used for both AIRTASKS and Work Unit Assignments. 

To help ensure efficient utilization of resources NAVAIRINST 
3900 . 8 A requires that all assignments involving T&E be coordinated 
with AIR-620, the T&E Projects Division of AIR-06. This is an 
important step in centralizing the control of field activity 
operations and promoting optimum use of their resources. On the 
other hand, it reduces the authority of the technical managers 
to deal directly with the field activities for work to be 
performed. For this reason the policy is followed reluctantly, 
if at all. 

The AIRTASK/Work Unit Assignment includes a section for 
detailed requirements. Quite often this section receives 
inadequate attention so that there is not a clear understanding 
between the originator and field activity personnel of what is 
required. This may be alleviated somewhat by referencing a work 
unit plan which has been generated by the field activity. This 
is satisfactory only if the person directing the work has read 
and understood the work unit plan and concurs in what it 
proposes. This is not always the case. Another method which 
may be used to promote understanding of the requirements is to 
require the field activity to submit a detailed program plan. 
Unfortunately, the instruction does not stress that this should 
be an iterative process, continuing until an agreement has been 
reached. All of this may be time consuming and may be intolerable 
on urgent projects. 
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2. Recommendations 



To avoid misunderstanding and to help preclude completion 
of the test program without satisfying the objectives, discussions 
should be held and proposed AIRTASKS/VORK UNITS should be reviewed 
prior to formal promulgation of the AIRTASK/WORK UNIT. This 
procedure should also be followed for detailed project plans 
when they are required. The discussions should include the user, 
the field activity, and AIR-620 representatives. 

B. TEST PLANS 

1 . Discussion 

Currently there is no standard method or format for test 
plans common to all field activities, nor is there a requirement 
to submit the test plan for review. This allows the field 
activities maximum latitude to establish procedures. This is 
particularly important in the area of flight safety where each 
activity assumes accident reporting responsibility and, therefore, 
must have the authority to establish their own safety procedures. 

2. Recommenda tions 

In spite of the need for maximum flexibility at the field 
activities, there is also a need for closer control by project 
personnel at headquarters. Field activity personnel are not 
always aware of important considerations which must be emphasized 
in the test program. To help ease this problem a base line format 
for test plans should be established which the field activities 
could modify to suit their own individual requirements. Test 
plans should be required by the AIRTASK/Work Unit Assignment to 
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be submitted for review. However, this provision must be used 
carefully to avoid loss of field activity flexibility and 
independence and to maintain an objective test program. 

C. REPORTING THE RESULTS OF T&E 
1 . Discussion 

For T&E to be of any value the results must be reported 
to the decision makers in a timely manner. Also, they must be 
reported in such a way that they may be easily and rapidly 
understood by someone who does not have a great deal of time to 
read lengthy reports. In the case of discrepant performance, 
the sooner the discrepancy is reported and action is taken to 
correct it, the less costly the correction is likely to be. The 
further the development or production process proceeds before a 
discrepancy is discovered, the more that must be corrected once 
it is discovered. 

No standard procedures have been established for 
reporting the results of T&E by the various field activities. 
Reports have been late and of little value because of the 
lateness. Information that has been used has been passed through 
briefings, letters, and messages. The report has only served as 
a record of what has been accomplished and is used very little 
as the basis for decisions. Under this system, only major 
discrepancies are reported in a timely manner. Minor problems, 
which taken together may have major impact, are generally not 
reported until the test program is complete when, in some cases, 
it may be too late to correct them. 



Procedures have not been developed at Headquarters to 
process incoming reports and make them readily available to 
anyone who needs them. They go to the code sponsoring the work 
and other codes designated by the field activity originating the 
report, but not necessarily to all offices requiring knowledge 
of the test results. No system exists that can be accessed by 
anyone needing the information to tell if a report has been 
submitted on a project, and, if so, how to locate the report. 

Field activity emphasis has been on current testing with 
reports being written and reviewed as the schedule allows. This 
is as much due to the reluctance of personnel to do the writing 
as it is due to any established policies. To overcome this 
reluctance, methods could be devised to make -the writing simpler 
and shorter. The use of deficiency reports is a good example of 
this. The deficiency reports are brief enough so that they can 
be easily generated with minimum detail and be modified as more 
is learned by submitting additional reports. They are submitted 
as problems are noted, not at the end of the test program. The 
use of deficiency reports, however, greatly increases the number 
of reports which must be tracked, including the actions taken 
as a result of the reports. This increases the requirement for 
a system to record all reports so that they may be accessed 
rapidly. 

2 . Recommendations 

To improve the usefulness of T&E reports, standard 
procedures should be established for their submission. This 
should Include a standard report format and should be based on 
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a system similar to that currently used for deficiency reports. 
Reporting requirements must be kept brief to allow timeliness. 

As a part of an overall T&E information system, procedures should 
be established to track all reports for easy access and to track 
actions taken as a result of the reports. 

D. WORK UNIT SUMMARIES AND FIELD ACTIVITY PLANS 

Navy MRTFB activities are required by Reference 11 to submit 
Work Unit Summaries for all current and expected projects to 
AIR-06 for use in planning and preparing the MRTFB budget and in 
the management of direct workload at the MRTFB activities. 

Included are appropriation and project information and estimates 
of the manpower and funding requirements of the project. Also, 
included is information on the background, objectives and progress 
of the project. Reference 11 also directs that workload sponsors 
provide written fiscal guidance to AIR-06 and the MRTFB activities. 
If the system works properly, the Work Unit Summaries should 
reflect this fiscal guidance. It has been the author’s experience 
that this is rarely the case; however, the instruction is 
relatively new and the situation may improve with time. 

Detailed back-up information for the manpower and cost 
estimate of the Work Unit Summary is provided by the Field 
Activity Plan. The manpower estimates are broken down by 
functional area and by type of personnel (military or civilian). 

The cost estimates are broken into manpower, material, flight 
hours, travel, contract, and funding to other activities. 

Obviously, this information must be balanced against the fiscal 
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guidance from the workload sponsors and, theoretically, the 
fiscal guidance should reflect the cost estimate if there is to 
be any agreement on what is to be done for how much. Again, 
there is little indication that this is the case, but hopefully 
the situation will Improve. 

E. COMPREHENSIVE INFORMATION SYSTEM 
1 . Discussion 

Many of the problems discussed to this point result from 
poor information flow. Much of this could be improved with 
revision and integration of the information systems used for T&E 
field activity management into a unified comprehensive information 
system. 

It is, therefore, worthwhile to examine the environment 
in which such an information system must operate, existing systems 
and the feasibility of establishing an information system which 
will allow efficient management of resources at the Headquarters 
and field activity levels. 

Currently various systems have been established at the 
field activities, primarily to manage their own internal 
resources and to provide accounting for the funding allocated 
for various projects. In terms of the overall NAVAIR organization 
these are primarily operations control systems with some lower 
level management control applications. To better control the 
test and evaluation process the TEMP is required for each major 
acquisition project. The TEMP outlines the testing required, 
the schedule, and required resources. Field Activity Plans and 
Work Unit Summaries outline the financial, manpower, and equipment 
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resource requirements and the approach to be taken. None of 
these documents, however, provides a complete picture of the 
capabilities, resources available, resources required, and 
workload of each of the field activities individually or in 
aggregate. The feasibility of establishing such a system should 
be investigated. 

The groundwork has now been established for the examina- 
tion of the environment of such a system in the NAVAIR organi- 
zation. Size of the organization does net appear as a deterrent 
to establishment of an information system. In fact, just the 
opposite is true. The large and diverse nature would seem to 
dictate that an information system be established if adequate 
planning of resource allocation and utilization is to be 
accomplished. 

The organizational structure of NAVAIR and its field 
activities may represent one of the most formidable barriers to 
the establishment of an effective information system. The 
organization is extremely decentralized. Each of the field 
activities has its own command structure with its own motives 
which in actuality may or may not align with the overall NAVAIR 
goals. Further the field activities report tc no single authority 
within headquarters. Nor do headquarters personnel deal with 
single points of contact at the field activities. Different 
headquarters offices are involved depending on whether the 
matters in question are associated with the conduct of individual 
projects or overall administrative control and support. The 
headquarters personnel associated with specific project areas 
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generally deal directly with field activity personnel who will 
be working on their projects. Because of these factors an 
extreme amount of coordination and cooperation would be required 
if a usable information system is to be established. 

Several factors effect the time frame within which an 
information system would need to be constructed. NAVAIR, in its 
role as developing agency, must deal with continuous change. 
Technology is changing rapidly and continuously, but also the 
regulations regarding systems acquisition and thus the methods 
that are required to be used are al30 continually changing. This 
often dictates that the interface with the field activities change 
from time to time. These changes, however, are usually relatively 
minor, and the relationship between NAVAIR and its field activities 
has not changed substantially in several years. Field activity 
control can probably be made more efficient and effective through 
the use of an information system; however, time constraints do 
not appear to be such that pressure exists for a development 
project that must be rushed to the exclusion of adequate planning. 

The organization in which the information system must 
operate is somewhat difficult to define. There are organizational 
segments which clearly have a large role as the prime contributors 
and users. However, there are other segments of the organization 
that will impact the operation whose participation is not clearly 
defined. The prime headquarters users of the system would be 
AIR-06; hov/ever, since a large portion of the interface between 
NAVAIR and the field activities includes the assignment of work 
by other parts of the organization, their inputs to the system 
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are also required. The relationships between these organiza- 
tional components is undergoing changes which will take some 
time to settle into focus. Until this occurs the organization 
cannot be considered mature and this must be recognized as a 
possible stumbling block to the establishment of a successful 
information system. 

NAVAIR Headquarters does not have in-house hardware or 
personnel to be utilized for the development and operation of 
the information system. Hardware and many of the personnel 
assets required exist at the field activities. Hardware could 
probably be made available on a time-sharing basis; however, 
whether or not personnel could be made available for the project 
would require some study. Also, it may be undesirable to have 
any one of the field activities established as the central 
location for information system data storage and analysis. A 
possible alternative is to contract the development and operation 
of the information system to an outside consulting organization. 
This is attractive since contractual mechanisms could be used to 
help control cost and schedule. The real problem arises in 
finding sufficient funding to adequately support the effort. 

Since the field activity control task is currently being performed 
without major difficulties being apparent at levels controlling 
the funds, a major effort will be required to sell the benefits 
of the system. One possibility to accomplish this might be to 
cite examples of past inefficiencies and their associated costs 
which could have been avoided with the proposed information 
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system in operation. A side benefit of thi3 would be to help 
define the areas toward which the information system should be 
directed . 

Attitudes about information systems are, unfortunately, 
generally hostile. Too often the same data have been required 
to be submitted several times in different formats to satisfy 
different information systems only to have the results of 
considerable effort go unused for any significant decisions 
regarding the operation of the organization. Expectations for 
an information system would probably be low; therefore, any 
demonstration of successful operation would probably come as a 
welcome surprise. One of the biggest problems is convincing 
the operations level people who make most of the inputs to the 
system that it is worth their while to spend the time and effort 
to make the inputs accurate. This could be significantly aided 
if the same information is required only once and if pa3t 
information submitted may be easily reviewed and updated without 
complete resubmission. Also, it is necessary that they be 
convinced that the results of their efforts serve some useful 
purpose. For these reasons it is important that the managers 
who are potential users of the system be convinced that it is 
worth their effort to stay involved in the development of the 
system beyond the establishment of the initial requirements. 

Assuming that in spite of the draw backs cited in the 
preceding paragraphs, it is decided that an information system 
for the controlling of field activity operations is worth the 
effort, it is possible to look at what might be some of the 
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elements of the Information system target. At least, the 
definition and some of the planning stages can be examined. 

Various systems have already been established at the 
field activity or operational level which may be modified or 
used intact as inputs to the information system. Some direction 
from headquarters as to the type of information required in the 
system is also required. For these reasons the parallel approach 
of Reference 14, where operational systems and planning and 
control systems are developed simultaneously, seems to be appro- 
priate for this information system development. Top down is 
ruled out because it would probably require more top management 
support than is available. This would be particularly true when 
attempting to integrate information from the various parts of 
the matrix organization. On the other hand, bottom up would be 
easier to accomplish, but due to differences between the field 
activities it is doubtful that a system with sufficient integration 
to be useful could be attained this way. 

There is little requirement for the field activities to 
exchange information between themselves; therefore, there is no 
requirement for a high degree of integration at this point. They 
should, however, be able to pass the same information in the 
same formats to headquarters for further aggregation and analysis. 
Although this is desirable it does not mean that nothing useful 
can be gained treating the information received from each of the 
field activities by itself. For example, aircraft requirements 
for current and future projects can be analyzed for each field 
activity individually. On the other hand, requirements for 
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funding for new instrumentation and facilities mu3t be compared 
for all activities to establish priorities for allocation of 
extremely scarce resources. 

Because of the psychological climate in which the system 
must operate, innovation and the attendant risk should be kept 
to a minimum. It is important to achieve early, if modest, 
successes to maintain and improve support for the system. One 
area where some innovation may be allowed is data base management. 
This technique offers considerable appeal in terras of establishing 
the system such that a given piece of data information needs to 
be entered into the system only once, which vrill make the system 
attractive to those who must feed it. 

2 . Recommenda tlon 

The priorities, specific functions, and goals of the 
projected information system require considerable in-depth 
investigation beyond the scope of this thesis. These could be 
developed using the preceding discussion and discussions with 
personnel who would be involved in the operation and use of the 
system. A study should be established involving, as a minimum, 
representatives of AIR-05, AIR-06, and the T&E field activities. 
High level attention must be maintained to ensure proper support 
for the system instituted as the result of this study. In the 
establishment of the information system a consultant familiar 
with the establishment of information systems should be hired to 
prevent many of the pitfalls which might not be seen by someone 
unfamiliar with the process of establishing an information system. 
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Emphasis in the development should be placed on ease of submission 
and updating of information and information requirements should 
be limited to those for which a valid use has been identified. 
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V. PROBLEM AREAS AMD OPPORTUNITIES 



A. ACTIVITY MISSION STATEMENT OVERLAP 
1 . Discussion 

There are four T&E field activities, listed in Table 1, 
under the administrative control of AIR-06. In addition signi- 
ficant NAVAIR T&E functions are performed at the Naval Weapons 
Center and the Atlantic Undersea Test and Evaluation Center, 
although they are not NAVAIR field activities. The missions of 
the field activities are stated in NAVAIR Instruction 5^51.82. 

Some areas are clearly within the mission of only one of the 
field activities. In other areas, however, there is considerable 
overlap in the stated missions. As an example, the mission 
statements of the two largest field activities, the Naval Air 
Test Center (NATO) and the Pacific Missile Test Center (PMTC) 
are quoted below. For NATC the mission is: 

"To conduct tests and evaluation cf aircraft weapons 
systems and their components. " 0 

For PMTC the mission is: 

"To perform development test and evaluation, develop- 
ment support, follow-on engineering, logistics and training 
support, for naval weapons, weapons systems and related 
devices; provide major range, technical and base support 
for Fleet users and ether DOD and government agencies. " D € 

This is only one example. There are overlaps with other 
activities also. NAVAIRINST 5^51.32 also states the specific 
functions to be performed by the field activities in the accomplish- 
ment of their missions. Here there is further overlap. 
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Table 1 



NAVAL AIR SYSTEMS COMMAND 
TEST AND EVALUATION FIELD ACTIVITIES 

Naval Weapons Evaluation Facility, Albuquerque, N.M. 

Naval Air Propulsion Center, Trenton, N.J. 

Naval Air Test Center, Patuxent River, MD. 

Pacific Missile Test Center, Point Mugu, C. 

The result of this overlap is a certain amount of 
wasteful duplication of facilities and capabilities and the 
competition between field activities cited by the Civilian 
Management Board/Middle Management Council £3 and discussed in 
the Introduction. Certainly, some of the duplication is 
necessary. All test ranges must have position measurement, 
communications, and data gathering and processing systems. Also, 
a certain amount of competition is inevitable and, in fact, 
beneficial because it helps to keep the field activities effi- 
cient and responsive to the needs of Headquarters. The key to 
whether the duplication is wasteful or not is in ohe amount of 
marketing done by the field activities for work and the degree 
to which the capacity of the field activities, as a group, is 
utilized. The marketing that accompanies the competition between 
the activities often results in waste because work is performed 
that is not really necessary. This would seem to indicate that 
capacity is not being utilized efficiently and some wasteful 
duplication does exist. This Is paradoxical because in many 
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cases the field activities claim to be overworked, underfunded, 
and understaffed, yet the inertia of the system causes them to 
continue to seek new work. In fact, at different levels in the 
organization there exist different perceptions of what the work 
level really is. One level of management may be actively pursuing 
new projects while another level is trying to discourage them. 

2 . Recommenda tlons 

It would appear that one way to decrease the wasteful 
results of competition is greater centralization of T&E project 
control in AIR-06 where a greater knowledge of the overall 
capabilities and workloads of the field activities exists. 

However, it is unlikely that this would work for two reasons. 
First, there are insufficient personnel in AIR-06 to assume the 
added responsibility and it is unlikely that additional personnel 
are available. Second, this is a reduction in the flexibility 
and authority of AIR-05 and would meet strong resistance from 
that quarter. The best solution is probably merely to encourage 
greater awareness of the problem and greater cooperation between 
AIR-05 and AIR -06 through the policies established in IIAVKATINST 
3960.10 and NAVAIRINST 39^0. 2A. Proposals have been made for 
the establishment of a facilities ard capabilities handbook which 
could be used to increase the awareness of all T&E facilities 
users, and particularly AIR-05* of the areas of expertise of the 
various T&E field activities. This would improve the situation 
and such a handbook should be published. 
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B. PLANNING AND BUDGETING FACILITIES IMPROVEMENTS 



1 . Discussion 

The Uniform Funding Policy and NAVMATINST 3960.10 
require that facilities improvements that support a specific 
project be paid for by funds from that project and that general 
improvements in capabilities be paid for by the MRTFB budget. 

In practice it is often difficult to place a given improvement 
in one category or the other. The result is that the "user" 
refuses to pay to improve the overall capability of the activity 
and AIR-06 refuses to pay for improvements to support a project 
and needed improvements are not accomplished. Further, there 
is no provision for facilities improvements which cannot be 
justified by an immediate application. This precludes long 
range planning and efficient modernization programs because 
accurate predictions of future projects are not available. 

The other difficulty with funding facilities improvements 
from the MRTFB budget, mentioned earlier, is the low priority 
of facilities improvements when compared with other demands for 
MRTFB funds such as salaries and other operating expenses which 
are relatively fixed and must be paid. When user funding falls 
short of budget estimates it is the facility improvements which 
inevitably suffer. Even when user funding is adequate, MRTFB 
funding must be held in escrow against possible shortfalls. This 
results in hurried spending at year end instead of a balanced 
program throughout the year. 
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2 . Recomtnenda tlcns 

To alleviate these problems and allow facilities 
improvements to be based on a coordinated long range plan, the 
funding of facilities improvements should be exempted from the 
Uniform Funding Policy and funded entirely from MRTFB funds 
except when large dollar value improvements are being considered 
for a specific project. In this case, "user" funding might be 
used to supplement MRTFB funding. The negotiation for "user" 
funding should be between AIR-06 and the program manager. The 
field activities should not be involved. Further, the IScM 
portion of the MRTFB budget, which funds the facilities improve- 
ments, should become a separate allocation not subject to other 
uses. This would result in a reduction in the flexibility of 
the field activities to utilize the MRTFB funds as required but 
it would help to ensure that needed improvements would be 
provided. 
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VI. SUMMARY OF RECOMMENDATIONS 



The recommendations which have been made generally fall Into 
one of three categories; changes in operating procedure , changes 
in budgetary procedure, and emphasis on implementing existing 
policies. These recommendations are summarized in the sections 
that follow. 

A. OPERATING PROCEDURE RECOMMENDATIONS 

Dialogue between users and the T&E field activities should 
be increased. Field activities should be considered a part of 
the NAVAIR team rather than as contractors. 

A single Integrated priority system for T&E field activity 
support should be established. 

Detailed discussions should be held to review AIRTASKS/WORK 
UNITS and detailed program plans prior to their promulgation. 

A standard format should be developed for test plans and 
they should be submitted for review by the work sponsor. 

Standard procedures should be developed for reporting the 
results of T&E. These should be based on simplified requirements 
which minimize the effort involved in preparing the report, 
which would improve timeliness. A report tracking system should 
be established at Headquarters to provide easy access to the 
reports and status of required actions. 

A study effort should be instituted which would result In 
the establishment of a NAVAIR T&E information system. 
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A facilities and capabilities handbook should be prepared 
to improve awareness of users of areas of expertise of the 
various T&E field activities. 

B. BUDGETARY PROCEDURE RECOMMENDATIONS 

The requirement to obligate all RDT&EN funds within the 
first year of their availability should be reexamined to determine 
if it causes more harm than good. 

Funding of facilities requirements, except in rare cases, 
should be paid for entirely from MRTFB funding. 

The I&M portion of the MRTFB budget should be a separate 
allocation, not subject to other uses. 

C. POLICIES WHICH REQUIRE FURTHER EMPHASIS 

Users of T&E field activities should submit written fiscal 
planning information to AIR-6103 per NAVMATINST 3960.10. 

Efforts for a strong I&M program should continue. 

To minimize wasteful duplication of facilities and capabil- 
ities and competition between T&E field activities, policies 
of NAVMATINST 3960.10 and NAVAIRINST 3960.2A must be followed 
more closely. 
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ELEMENTS OF THE MAJOR RANGE AND TEST FACILITY BASE [S] 
NAVY 

Pacific Missile Test Center 

Atlantic Undersea Test and Evaluation Center 

Naval Air Test Center 

Naval Air Propulsion Center 

Naval Weapons Center 
(T&E Portion Only) 

Atlantic Fleet Weapons Training Facility 

AIR FORCE 

Eastern Test Range 

Space & Missile Test Center 

Satellite Control Facility 

Arnold Engineering Development Center 

4950th Test Wing 

Tactical Fighter Weapons Center 

Flight Test Center 

Armament Development and Test Center 
Air Defense Weapons Center 



ARMY 

Cold Regions Test Center 
Tropic Test Center 
Yuma Proving Ground 
Jefferson Proving Ground 
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White Sands Missile Range 
Kewajalein Missile Range 
Dugway Proving Ground 
Aberdeen Proving Ground 

NOTE: The Naval Weapons Evaluation Facility is a NAVAIR T&E 

field activity not included in the MRTFE. 
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APPENDIX 3 



DEFINITIONS OF ACQUISITION CATEGORIES 0 . 2 ] 



ACAT I: Projects designated by the Secretary of Defense 

(SECDEF) or Deputy Secretary of Defense (DEPSECDEF) having a 
nominal dollar value in excess of $75M RDT&E or $200M Production 
and such other lesser programs as may be so designated by SECDEF 
or DEPSECDEF. 

ACAT II: Projects designated by a Defense Systems Acquisition 

Review Council (DSARC) principal, SECNAV or CNO having a nominal 
dollar value in excess of $20M RDT&E or $50M Production and such 
other lesser programs as may be recommended by the Chief of 
Naval Material (CHNAVMAT), CP-090, 0P-0S8 or the program sponsors 
(DCNO/DMSO) . 

ACAT III: Projects designated by the OPNAV sponsor, having 

a nominal dollar value in excess of $5M RDT&E or $ 20 M Production 
and such other lesser programs recommended by CHNAVMAT, OP-090, 
OP-O 98 , or the DA. Programs below the ACAT III threshold will 
normally be designated ACAT III if they: 

(a) Directly affect the military characteristics of 
ships, aircraft or other combat units; 

(b) Require Operational Test and Evaluation (OT&S) to 
support key program decisions; or 

(c) Require Fleet RDT&E support. 

ACAT IV: Projects not designated as ACAT III or higher 

which meet any one of the following criteria: 

(a) Have a unit cost of more than $10,000, net 
including management and support costs; 

(b) Have a total project cost of $1M (R&D plus 
production) excluding shore based training devices and shore 
support equipment not used in direct support of deployed 
systems; 

(c) The hardware required formal aoproval for service 
use (ASU). 
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APPENDIX C 



DIRECTIVES AND INSTRUCTIONS 



DOD Directive 3200.11, Use, Management and Operation of 
Department of Defense Major Ranges and Test Facilities, 

June 18, 1974. 

DOD Directive 5000.1, Major System Acquisition, January 18, 1977. 

DOD Directive 5000.2, Major System Acquisition Process, January 
18 , 1977. 

DOD Directive 5000. 3* Test and Evaluation, January 1, 1973- 

SEC NAVINST 5000.1, System Acquisition in the Department of 
the Navy. 

OPNAVINST 3900. 25A. 

OPNAVINST 3980.10, Test and Evaluation. 

OPNAVINST 5440. 47D, Mission and Functions of Operational Test 
and Evaluation Force (OPTEVFOR), 6 March 1973. 

NAVMATINST 3910.15A. 

NAVMAT INST 3960 . 6 A, Test and Evaluation. 

NAVMATINST 3980.8, Land-Eased Test Site (LETS) Selection Policy, 

1 September 1976. 

NAVMATINST 3960.10, Policy and Procedures for Workload Manage- 
ment at the Navy Major Range and Test Facility Base (MRTFB) 
Activities, 3 February 1978. 

NAVMATINST 5450. 31> Consolidation of Navy Ranges, Test 
Activities, and Target Test Resources, 29 March 1976. 

NAVAIRINST 3900. 8 A, AIRTASK and Work Unit Assignments to Field 
Activities, 22 March 1977. 

NAVAIRINST 3960. 2A, Test and Evaluation, 11 August 1978. 

NAVAIRINST 5451.82, Naval Air Systems Command (NAVAIR) Test and 
Evaluation Activities; Mission and Functions of, lo December 1975. 
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